fix(runtime): reclaim credential-home locks whose owner names no process - #229
Open
rohanpoudel2 wants to merge 4 commits into
Open
fix(runtime): reclaim credential-home locks whose owner names no process#229rohanpoudel2 wants to merge 4 commits into
rohanpoudel2 wants to merge 4 commits into
Conversation
`recoverStaleCredentialHomeLock` consulted `process.kill(pid, 0)` for any `owner.json` whose `pid` was a number. POSIX gives two of those numbers special meanings: 0 signals the caller's own process group and -1 every process it may signal, so both always succeed and always report a live owner. A fractional or out-of-range value makes `process.kill` throw `ERR_INVALID_ARG_TYPE`, which is neither ESRCH nor EPERM and was rethrown raw out of a public API. Because the age check sits in the `else` branch, a lock naming any of these values was also exempt from it, so `acquireCodexSecurityCredentialHomeLock` waited on it forever at a 25 ms poll with no message and no timeout. Only consult `process.kill` for a positive safe integer. Anything else is an owner that cannot be identified, and is now treated like a missing one, so the existing 30 s age check reclaims the lock. This does not address a genuinely reused pid, which needs a heartbeat rather than a liveness probe and is a larger design change. That part is described in the issue. Refs openai#228
Collaborator
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8f0bc4d3a3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Security review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
`process.kill` narrows its pid argument to a 32-bit signed integer and throws ERR_INVALID_ARG_TYPE when the value does not survive that round trip. `Number.isSafeInteger` still admits positive integers above that range, such as 2147483648, so an aged `owner.json` naming one reached `process.kill`, and the argument error - being neither ESRCH nor EPERM - escaped `recoverStaleCredentialHomeLock` and failed the acquisition outright instead of reclaiming the malformed lock. Reject any owner pid above the maximum `process.kill` accepts so it is treated as unidentifiable, letting the age check reclaim the lock, and cover the gap between the pid range and the safe-integer range in the stale-lock test.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #228
Problem
recoverStaleCredentialHomeLockconsultedprocess.kill(pid, 0)for anyowner.jsonwhosepidwas a number:POSIX gives two of those numbers special meanings —
0signals the caller's own process group and-1every process it may signal — so both always succeed and always report a live owner. A fractional or out-of-range value makesprocess.killthrowERR_INVALID_ARG_TYPE, which is neitherESRCHnorEPERMand was rethrown raw out of a public API.The age check sits in the
elsebranch, so a lock naming any of these values was also exempt from it.acquireCodexSecurityCredentialHomeLocktherefore waited on such a lock forever, at a 25 ms poll, with no message and no timeout — while every other stale path (missing or corruptowner.json) has a 30 s escape hatch.Measured against the public API on a lock aged 24 h, with a 1.5 s abort as the only way out:
Change
Consult
process.killonly for an integer that is positive and inside the range it accepts:Anything else is an owner that cannot be identified, and is now treated exactly like a missing one, so the existing 30 s age check reclaims the lock. All seven cases above now acquire in 2–3 ms.
The
EPERMhandling is unchanged and still means "the process exists but we may not signal it".The upper bound is not a guess. Node's guard is
pid != (pid | 0)— a ToInt32 round trip in its own JS layer, before any syscall — so the accepted range is exactly signed 32-bit, and it is a Node property rather than an OS one. Measured on Node v24.11.1:Why not
Number.isSafeIntegeraloneThat was this PR's first shape, and it leaves a gap: every integer from
2147483648up to2 ** 53 - 1is a safe integer thatprocess.killstill rejects. Anowner.jsonnaming one reachedprocess.kill, and the argument error — neitherESRCHnorEPERM— escapedrecoverStaleCredentialHomeLockand failed the acquisition outright instead of reclaiming the malformed lock:Number.isSafeIntegeronlyNumber.isInteger+ bound21474836482 ** 53Number.isIntegerreplacesNumber.isSafeIntegerin the final predicate because<= MAX_PROCESS_IDalready subsumes it — every integer in the pid range is a safe integer.Why not an absolute age ceiling on held locks
A genuinely reused pid is still indistinguishable from the original owner, so a lock left behind by a
SIGKILLed scan is still never reclaimed. I left that out on purpose:#runbefore runtime initialization, released in the outerfinally), and scans legitimately run well over an hour (Standard full-repository scan shows only generic heartbeat for 40+ minutes — expected behavior? #70, Detect and warn from HEAD drifting earlier #164). A plain absolute age ceiling would therefore let one scan steal a running scan's lock — strictly worse than the current behaviour.#228 documents that part, including why it is close to guaranteed in the shipped container (
Dockerfileputs the state dir under the/outputbind mount, andcompose.yamlsetsinit: true, so container pids restart from 1 each run and a leftover low pid collides). Happy to implement the heartbeat if you tell me which shape you want.Impact, stated plainly
Narrow. The only thing that changes is which
owner.jsonvalues are believed to name a live process. A lock whose pid is a real, in-range pid behaves exactly as before: live still means live,EPERMstill means live,ESRCHstill means reclaimable. The values whose handling changes —0,-1, fractional, anything past the int32 range — are never written by this code, which always writesprocess.pid. They arrive only from a corrupted or hand-edited lock, which is precisely the case that used to hang the scan forever or throw an argument error out of a public API.Verification
recovers credential-home locks whose owner names no processwalks[0, -1, 0.5, 2 ** 31, 2 ** 53], ages the lock past the 30 s threshold, and bounds the acquisition with anAbortControllerso a regression fails the test in 5 s instead of hanging it.2 ** 31is2147483648, the first valueprocess.killrejects.typeof ownerPid === "number"only):0 pass / 1 fail—AbortError: The operation was aborted.after 5003.13 ms.Number.isSafeInteger(ownerPid) && ownerPid > 0):0 pass / 1 fail—ERR_INVALID_ARG_TYPE: The "pid" argument must be of type number. Received type number (2147483648)after 11.52 ms.1 pass / 0 failin 54 ms.Full suite with the change: 739 pass / 5 skip / 0 fail across 34 files, 5169
expect()calls. With the range bound reverted the same command reports736 pass / 5 skip / 3 fail; only one of those three is this test, the other two are thecli.test.tsnpm-style-bin symlink cases, which fail intermittently on this machine under parallel load and pass on re-run with the change restored.pnpm run typesandpnpm run formatboth exit 0.